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Abstract 

We describe a certification assistant to support formal safety proofs 
for programs. It is based on a graphical user interface that hides the low- 
level details of first-order automated theorem provers while supporting 
limited interactivity: it allows users to customize and control the proof 
process on a high level, manages the auxiliary artifacts produced during 
this process, and provides traceability between the proof obligations and 
the relevant parts of the program. The certification assistant is part of a 
larger program synthesis system and is intended to support the deploy- 
ment of automatically generated code in safety-critical applications. 


1 Introduction 

Program verification remains one of the most promising applications of theorem 
proving, and both fully automatic and interactive provers have been used in 
verification projects. However, program verification has not lived up to its early 
promises and is not yet applied routinely in software development. This has 
a variety of reasons, ranging from the technical difficulties the task still poses 
for theorem provers, to problems in designing appropriate interfaces, and the 
cultural changes that are necessary in the software development process itself. 

In this paper, we describe a user interface we have developed, for the appli- 
cation- of fully automatic -first-order theorem provers (ATPs)- in formal program 
certification, a limited and thus more tractable variant of full program verifica- 
tion. It uses the same basic technology but is concerned only with safety-relevant 
aspects of a software system rather than the complete system behavior. This 
limitation enables the successful application of ATPs. Program certification is 
based on the idea that mathematical proofs of the individual safety properties 
can be regarded as certificates which can be subjected to external scrutiny and 
related to the relevant safety-critical parts of the software system. This par- 
ticular purpose of the proofs requires a dedicated application-oriented interface 
rather than a proof-oriented interface — in effect, we need a certification assistant 
rather than a proof assistant. 
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The role as certification assistant puts the user interface under the influence 
of two competing design principles. On the one hand, it has to “hide” the the- 
orem provers from unsuspecting software engineers. On the other, it has to be 
open in order to provide coarse-grained control of the certification process, to 
maintain traceability between the different artifacts (in particular, source code 
and verification conditions), and to ensure trust in the entire certification pro- 
cess. As a consequence of this duality, the interface cannot completely separate 
the ATP from the rest of the certification environment. In particular, it needs 
to support the interpretation of both the input to and the output from the ATP 
in terms of the application. 

Our work on certification emerged from an ongoing project on automated 
program-synthesis. We have developed two synthesis systems for the domains-of- 
scientific data analysis [FS03] and state estimation [WS05], which can generate 
code for safety-critical application areas like spacecraft guidance, navigation, 
-a-nd-eeBt-reP-Fr-©eess-standards-su6h-as-DO-4-7§B4RTC92j^iiow:even r xequire_that 
all safety-critical software be certified to a high degree of confidence. Our goal is 
thus to integrate our synthesis tools with a dedicated certification environment 
so that end-users can understand and trust the generated code more easily. 
We adopt a browser paradigm so that users can inspect the code and interact 
with the underlying prover, while being shielded from the low-level minutiae. 
Although this is still work in progress, we believe that it offers potential for 
increasing acceptance of code generators in safety-critical domains at NASA. 

In Section 2, we introduce our automated synthesis and verification systems 
and describe the certification problem we address. The system architecture has 
a direct bearing on the certification interface, which is described in Section 3. 
Although the system is fully automatic, users have the option of controlling the 
proof process by selecting and parameterizing, different provers, and inspecting 
the logs of prover sessions, including the proofs themselves. This is described 
in Sections 3.1 and 3.2, respectively. In Section 3.3, we describe the verification 
condition browser, which is used to relate proof obligations to the synthesized 
code. Section 4 describes related work on interfaces for prover-based verification, 
and Section 5 outlines our future plans. 

2 Background and System Architecture 

Figure 1 shows the overall architecture of our extended program synthesis sys- 
tem, which comprises three classes of components: the original synthesis system, 
the certification extensions, and corresponding document generation extensions. 
We will describe these components in some more detail in the following. 

2.1 Program Synthesis 

Traditionally, program synthesis has followed the proofs-as-programs paradigm: 
the problem is specified as a conjecture in a suitable logic, an interactive theorem 
prover like Coq, Isabelle, or NuPRL is used to construct a proof, and a func- 
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Figure 1: Certifiable program synthesis: System architecture 


tional program is extracted from that proof and then translated into the target 
environment. However, this traditional, purely deductive approach to program 
synthesis is notoriously difficult to scale up to large problems (cf. [AB01]) and 
full automation has remained elusive. We thus follow a schema-based, synthesis 
approach that combines deductive reasoning with techniques from generative 
programming. Most of the components described here are hidden inside the 
synthesis system box in Figure 1 , 

Problem Specifications. Since schema-based synthesis does not require 
a logical conjecture as starting point for a proof, the code derivation can be- 
gin with a specification in a more application-oriented domain- specific language. 
The details of the language obviously depend on the domain of- the synthesis 
system, but in general it combines some target language constructs (e.g., dec- 
larations) with established scientific and engineering notations (e.g., differential 
equations). This allows a concise and fully declarative formulation of the prob- 
lem together with some details of the desired configuration and architecture of 
the code to be generated. 

Schemas. A schema is a parameterized code fragment (i.e., template) to- 
gether with a set of constraints that determine whether the schema is applicable 
and how the parameters can be instantiated. The constraints are formulated as 
conditions on a problem model, which allows the problem structure to directly 
guide the application of the schemas and thus constrain the search space. The 
parameters are instantiated by the synthesis engine, either directly on schema 
application or by recursive calls with a modified problem. The schemas are or- 
ganized hierarchically into a schema library which further constrains the search 
space. Schemas represent both fundamental building blocks (i.e., algorithms) 
and solution methods (i.e., theorems) of the domain; they are thus similar to 
the lemmas used in interactive systems but they can contain explicit calls to a 
meta-programming kernel in order to construct the code fragments. 
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Symbolic Computations. Symbolic computations are used to support 
schema instantiation and code optimization. The core of the symbolic subsystem 
is a small rewrite engine which supports associative-commutative operators and 
explicit contexts. It thus allows contextual rules as for example x/x — >ci-x^o 1 
where —>c i-i^o means “rewrites to, provided x ^ 0 can be proven from the cur- 
rent context C.” Expression simplification and symbolic differentiation, similar 
to those in Mathematica, are implemented on top of the rewrite engine. The 
basic rules are straightforward; however, vectors and matrices require careful 
formalizations, and some rules also require explicit meta-programming, e.g., 
when bound variables are involved. 

Intermediate Code. The code fragments in the schemas are formulated in 
an-imperative intermediate language-.- - This is essentially a “sanitized” variant of 
C (i.e., no pointers, side effects in expressions etc.); however, it also contains a 
number of domain-specific constructs like vector /matrix operations, finite sums, 
-a-nd-eenvergence-loeps, — 

Optimization. Straightforward schema instantiation and composition pro- 
duces suboptimal code; worse, many of the suboptimalities cannot be removed 
completely using a separate, after-the-fact optimization phase. Schemas can 
thus explicitly trigger large-scale optimizations which take into account infor- 
mation from the synthesis process. For example, all numeric routines restructure 
the goal expression using code motion, common sub-expression elimination, and 
memoization; since the schemas know the goal variables, no dataflow analysis is 
required to identify invariant sub-expressions, and code can be moved around 
aggressively, even across procedure borders. 

Code Generation. In a final step, the optimized intermediate code is 
translated into code tailored for a specific run-time environment. We currently 
have code ' generators for the Octave and Matlab environments, and. can also 
produce standalone Ada, C, and Modula-2 code. Each code generator employs 
one rewrite system to eliminate the constructs of the intermediate language 
which are not supported by the target environment ( “desugaring” ) and a second 
rewrite system to clean up the desugared code; most rules are shared between 
the different code generators. 

AutoBayes and AutoFilter. So far we have built two domain-specific 
synthesis systems following the schema-based approach outlined above. Auto- 
Bayes [FS03] works in the scientific data analysis domain and generates param- 
eter learning programs, while AutoFilter [WS05] generates state estimation 
code based on variants of the Kalman filter algorithm. Both systems share a 
large common core (e.g., symbolic subsystem, certification subsystem, and code 
generators) but have their individual schema libraries. They are implemented in 
SWI-Prolog and together comprise approximately 100 kloc. Both systems work 
fully automatically and can generate code of considerable size and complexity 
(approximately 1500 loc with deeply nested loops) within a few seconds. 


4 



2.2 Uert incation 


Unlike purely deductive approaches, schema-based synthesis cannot ensure “cor- 
rectness-by-construction” . Since formally verifying the entire system is unfeasi- 
ble, we instead validate each generated program individually; furthermore, we 
concentrate on specific aspects of program safety (e.g., memory safety). The 
core idea is that the schemas can be extended to simultaneously generate code 
and all required annotations such that a verification condition generator can 
produce proof obligations which are then discharged using an automated theo- 
rem prover. The proofs, which can be validated by an automated proof checker 
or prepared for human inspection, then serve as certificates. This approach is 
tractable because the synthesis system has full knowledge about the form the 
generated code will take and the specific safety aspect that is to be certified, so 
that it can generate the appropriate annotations. However, our certification ap- 
proach is not necessarily tied to synthesis and the annotations could in principle 
also be added manually. 

Safety Policies. A safety policy is a set of Hoare-style proof rules and auxil- 
iary definitions which are designed to show that “a program does not go wrong,” 
i.e., satisfies the safety property of interest [WSF02, DF03]. Safety policies can 
be used to enforce both language-specific properties which can be expressed in 
terms of the constructs of the underlying programming language, itself, and are 
thus sensible for any program in the language, as well as domain- specific prop- 
erties, which typically relate to high-level concepts outside the language (e.g., 
matrix multiplication). 

We currently support five different safety policies. Array-bounds safety re- 
quires each access to an array element to be within the declared bounds of the 
array. Variable initialization-before-use ensures that each variable or individual 
array element has been assigned a defined value before it Is used. Both are 
typical examples of language- specific properties. For the data analysis domain, 
we can guarantee vector-norm safety (i.e., probability vectors add up to one), 
and for the state estimation domain we can check proper sensor input usage 
(i.e., all input variables are used in the computation of the filter output) and 
matrix symmetry (i.e., covariance matrices are not skewed). 

Annotated Code. The annotations are part of the schema and thus are 
instantiated in parallel with the code fragments; further annotations are intro- 
duced by the desugaring steps of the code generation phase. The annotations 
contain local information in the form of logical pre- and post-conditions and 
loop invariants, which is then propagated through the code. 

VCG. The fully annotated code is then processed by a weakest precondition 
verification condition generator (VCG), which applies the Hoare-rules of the 
safety policy in order to generate verification conditions (VCs). The VCG has 
been designed to be “correct- by- inspection” , i.e., to be sufficiently simple that 
it is straightforward to see that it correctly implements the rules of the logic. 
Hence, it does not implement any optimizations, such as structure sharing on 
the VCs or even apply any simplifications. As usual, the VCG works backwards 
through the code and verification conditions are generated at each line that can 
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potentially violate the safety policy. 

Simplification. By design of the VCG, the generated VCs are quite com- 
plex; hence, they need to be simplified before they can be discharged by an 
ATP. The certification extension thus re-uses the rewrite engine of the synthe- 
sizer together with a dedicated set of rewrite rules. Details can be found in 
[DFS04a] . 

ATP. For our purposes, an ATP is a search procedure which applies the 
inference rules of its calculus until it either finds a proof or fails because none 
of the rules are applicable. In order to handle extra-logical operations (as, for 
example, arithmetic functions), the ATP needs an additional domain theory 
that specifies their intended meaning as axioms. The provers use a set of core 
axioms, together with a collection of dynamically generatedaxioms-, -depending 
on the particular proof task. 

Proof Checking. As an alternative to formally verifying the ATPs, they 
-ean-be-e-xt-ended-to-generate-suSc iently detailed^achsnEhich-c an then b edim 
dependently checked by a small and thus verifiable algorithm. However, due to 
the lack of a standardized format, (and various other reasons [DFS04b]) there 
are almost no proof checkers for high-performance ATPs, in contrast to the sit- 
uation for interactive higher-order provers. We have linked our system to the 
only exceptions we are aware of: the IVY system [MSOO], which is based on 
Otter, and the GDV verifier [SB05]. 

Trusted Components. Similarly to proof carrying code [NL98], we dis- 
tinguish between trusted and untrusted components, shown in Figure 1 in red 
(dark grey) and blue (light grey), respectively. Components are called trusted — 
and must thus be correct — if any errors in them can compromise the assurance 
provided by the overall system. Untrusted components, on the other hand, are 
not. crucial to the assurance, because their results are double-checked by at least 
one trusted component. In particular, the correctness of the certification system 
does not depend on the correctness of the two largest components: the synthe- 
sizer, and the theorem prover; instead, we need only trust the safety policy, the 
VCG, the domain theory, and the proof checker. 

2.3 Document Generation 

The basic idea behind our certification approach can also be extended to human- 
readable documentation. The schemas contain text . templates that are instan-. 
tiated and composed together with the code fragments; these auto-generated 
comments explain selected parts of the algorithm, give detailed derivations of 
mathematical formulas, and relate program constructs and variable names back 
to the specification. 

In' addition to the commented code, we can also generate 'k standardized 
software design document that contains interface descriptions, administrative 
information (names of files, versions, etc.), specific input and output constraints, 
and synthesis and compiler warnings. The document is hyper linked to the input 
specification, the code, and any other intermediate artifacts generated during 
synthesis. Since the design document is generated at synthesis time, it can 
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Figure 2: Certification assistant: start-up view 


include design details which would be difficult, if not impossible, to infer after 
the code has been generated. 

3 The Certification Assistant 

The certification assistant serves three main purposes. First, it allows users to 
customize and control the proof process on a high level. Second, it manages 
the auxiliary artifacts produced by this process. Third, it provides traceability 
between the VCs and the relevant parts of the program. 

The interface mainly uses straightforward HTML as underlying technology. 
This is augmented with some scripting code to support the VC-browsing de- 
scribed in Section 3.3. The certification assistant consists of a few supporting 
shell scripts to control the provers, some boilerplate CGI and HTML code and 
a number of PHP files that are auto-generated together with the target code; 
in total, this amounts to approximately 2000 lines. The generation of the PHP 
files and the customization of the certification assistant itself is triggered by a 
simple command line option of the synthesis system. . . ..... 

Figure 2 shows the startup view of the certification assistant, after the code 
and supporting files have been generated. The window is split into three different 
areas. The left half contains a hyperlinked version of the generated code; the line 
numbers are used as labels by the VC linker. The right half contains a simple 
prover control panel and the list of verification conditions below that. Initially, 
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Figure 3: Certification assistant: updated view with prover results 

no information is available about the proof status of any of the verification 
conditions. 

3.1 Prover Control — 

In contrast to interactive proof systems like Coq, Isabelle, or NuPRL, the user 
interaction here is only concerned with the parameterization of the proof pro- 
cess, but not with application of individual tactics, and the prover control panel 
reflects this restricted interaction style. Different drop-down menus allow the 
user to select the theorem prover and choose between various predefined ax- 
iomatizations of the domain theory that are to be used for the proof attempts, 
and the level of evidence (i.e., proof status, prover logs, or full proofs) that the 
prover is required to supply. In addition, the user can specify the time limit 
for an individual task and any prover-specific parameters that will be passed 
along unchecked. The certification process is then started by selecting any or 
all of the verification conditions from the fist and sending the request, which 
the. certification assistant passes to the selected prover. 

Figure 3 shows the view updated with the prrtver results: For each verifica- 
tion condition, the current proof status is displayed, together with the elapsed 
proof time, and a link to its location in the source code (cf. Section 3.3). 
If evidence is produced, the status contains a link to the evidence (cf. Sec- 
tion 3.2). If some proof attempts fail (e.g., the verification condition quater- 
nion.dsl-init-0034), the user can resend a request with different settings or 
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with a different ATP. 

The certification assistant serves as common interface to different first-order 
ATPs in a similar spirit to how Proof General [AspOO] serves as common inter- 
face for different interactive higher-order provers. However, due to the limited 
interaction and the black-box style integration, the protocol requirements to 
link to a prover are much simpler. In particular, we can use the TPTP syn- 
tax [SS03] as common notation which is understood by most targeted ATPs 
and can be easily translated for the others. Hence, only straightforward con- 
trol scripts are required for the integration; we are currently also in the process 
of replacing these by the control scripts used in the annual ATP system com- 
petition (CASC) [Sut04]. The certification assistant currently integrates seven 
different ATPs (DCTP [LS01],.E [Sch02], E-Setheo [MI+97], Gandalf [Tam97], 
Ivy [MSOO], Otter [MW97], Spass [Wei03], and Vampire [RV02]), most of them 
in different versions, that run on the local server. In addition, it also provides 
a remot e link t .n the System OnTPTP proof sprvpr pit the Univ ersit y of M i ami 
[Sut05] , which then acts as a trusted prover component repository. 

3.2 Certificate Inspection and Visualization 

The certification assistant also provides access to the auxiliary artifacts that are 
produced during the certification. This includes the intermediate stages in the 
processing chain (generated axioms, clausal normal form etc.), prover log files, 
and actual proofs, depending on the required level of evidence. These artifacts 
can support, or in the absence of a proof collectively serve as, the certificate, 
and can be inspected as raw text files, or using third-party tools, e.g., the GDV 
derivation verifier [SB05] and the proof visualizer from the TPTP tool suite 
[SS03], , ________ _____ 


3.3 VC Linker and Browser 

A VC can fail to be proven for a number of reasons. First, there may of course 
be an actual safety violation in the code. Second, the (generated) annotations 
may be insufficient or wrong. Errors can come from any part of the schema, 
or from the propagation phase: an annotation might not be propagated far 
enough, or it might be propagated out of scope. Third, the theorem prover 
may time-out, either due to the size and complexity of the VC, or due to an 
incomplete domain theory. For certification purposes, however, it is important 
to distinguish between unsafe programs and any other reasons for failure, and in 
the case of genuine safety violations, to locate the unsafe parts of the program. 

However, manually tracing the VCs back to their source is quite difficult as 
the - verification process is inherently logically complex. The VCs-can- become 
very large and go through substantial structural simplifications, after which 
they are typically [DFS04a] of the form hyp 1 A • • • A hyp n =>- cone. Here, a 
hypothesis, hyp, stems either from a loop invariant, an index bound, or a propa- 
gated annotation, and the conclusion, cone , is either derived from an annotated 
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Figure 4: Certification assistant: linking from VC 


assertion or from a generated safety condition. Hence, a single VC can depend 
on a variety of information distributed throughout the program. 

In order to support tracing between the VCs and the source code, the VCG 
adds the appropriate l oca tion in forma tion to the formulas it constructs as it 
processes a statement at a given source code location. We currently use simple 
line numbers as locations rather than individual subterm positions [Fra96]. 

The source locations need to be threaded through all stages of our certifica- 
tion architecture (Figure 1), and, in particular, through the simplifier. We have 
thus extended the rewrite rules used for simplification to preserve the associated 
labels through the rewrite process, similar to the labeled rewrite rules used in 
the Simplify prover [DNS03]. This approach requires careful “rule engineering” 
to maintain the relevant location information while minimizing the scope of the 
labels and thus preventing the introduction, of too. much. noise, into the linking 
process. However, since each VC is generally linked to multiple statements, 
the location information for the entire program needs to be maintained, even if 
we just want to know whether a single line in the code satisfies a given safety 
property. 

Figure 4 shows how the tracing information can be used to support the 
certification process. A click on the source link associated with each verification 
condition prompts the certification assistant to highlight all affected lines of the 
code. A further click on the verification condition link itself displays the formula, 
which can then be interpreted in the context of the relevant program fragments. 
This helps domain experts assess whether the safety policy is actually violated, 
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Figure 5: Certification assistant: linking from code 


which parts of the program are affected, and eventually how the violation can 
be resolved. This traceability is also mandated by relevant standards such as 
DO-178B [RTC92]. 

_ In practi c e, s a f ety checks are often carried out during code reviews [ NS 04], 

where reviewers look in detail at each line of the code and check the individual 
safety properties statement by statement. Fortunately, linking works in both 
directions: clicking on a statement or annotation displays all VCs to which it 
contributes (i.e., which are labeled with its line number). Figure 5 show's the 
result of clicking on the label for fine 220; the unproven verification condition 
implies that this line of code has not been cleared yet. 

4 Related work 

There are a number of program verification systems which use theorem prov- 
ing, both automated and interactive, as the underlying technology. However, 
the provers are hidden to varying degrees. Fully automated provers are typi- 
cally used as black-box components in a tool chain thaLcontrols the ..verification 
process and determines the form of the interface. Often, the interfaces follow 
a compiler style, with command line parameters as inputs and error messages 
as outputs (e.g., ESC/Java [FL + 02, DNS03]), although an integration with a 
graphical software development environment is also possible [AB + 03]. Inter- 
active systems, on the other hand, typically use expressive higher-order logics 
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to model the entire software within the system, relying on the built-in prover 
interface to directly control and interpret the verification process. 

Perfect Developer [Cro03] is a combined software development and verifica- 
tion environment. The user writes annotated programs in an intermediate lan- 
guage, where the annotations express a correctness specification in the design- 
by-contract methodology. The programs can then be translated into a variety 
of target languages, or analyzed using Hoare logic and an automated theorem 
prover. 

Caveat [BC + 03] offers similar analysis capabilities, although it operates di- 
rectly on source code (as opposed to an intermediate language), with the ad- 
vantage that bugs can then be directly located. The tool tries to verify that the 
-eodeis free of the usual range of safety violations (division by zero, null deref- 
erencing, array out of bounds), based on Hoare logic. If the automated prover 
fails, the user can start an interactive proof process using Caml as scripting 
-language, — The-4eveloperst^ta±ed-aim-isHo-iiicrease_the_acQp£Ljfoji±it.erac.tiy- 
ity and control of proof process. Users are also able to manually insert code 
annotations as required. For debugging purposes, Caveat lets users interpret 
failed proof obligations, and relate them to their origin. It can also generate 
counter-examples . 

There are very few other approaches to providing explicit traceability for a 
program synthesis system. Van Baalen and others [vBR + 98] use origin tracking 
[vKT93] to indicate how statements in synthesized code relate to the initial prob- 
lem specification and domain theory. They later built on this to present a doc- 
umentation generator and XML-based browser interface that generates an ex- 
planation for every executable statement in the synthesized program [WB + 01]. 

The Proof General [AspOO] generic prover interface aims to shield users from 
the-low-level. details, of. using a.theorem prover. It offers a cu stomizabl e u ser 
interface, while adding functionality on top of that provided by the underlying 
prover. Our aims are similar, although we target a different abstraction level. 
There are also several developing higher-order proof networks (e.g., Mathweb 
[FH + 99]), but it is not yet clear what role they should play in a certification 
assistant. The SystemOnTPTP proof service [Sut05], which we use, can be seen 
as a first-order automated equivalent of these. 


5 Future Work 

Our current efforts focus on extending the certification system in a number 
of areas. We are currently extending the markup the VCG adds to the proof 
obligations in two different ways. First, we are replacing the plain line numbers 
'currently used as- locations by -individual subterm positions, similar to-{Fha96]. 
This allows a more precise tracing. Second, and more significantly, we are 
extending the VCG to augment the VCs with semantic information concerning 
the interpretation of the different parts of the formula (e.g., “loop invariant 
asserted in line X”). This simplifies the interpretation of the VCs for debugging 
purposes and can also be used to generate high-level descriptions of the VCs. 
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Since both extensions change the structure of the location information, we now 
need to correspondingly extend the interface to make use of this information. 

We are also extending the interface to include specifications and design doc- 
uments, thus combining our work on certification with automated safety and 
design document generation [DV04]. Eventually, we aim to provide full and 
seamless traceability between specification, design documentation, code, and 
certificates. 

Our broader aim is to develop a certificate management system along the 
lines of the Programatica project [The03], This will enable a wide range of 
additional capabilities, such as support for manual sign-offs on code fragments 
that violate stated safety policies. 

Acknowledgments. Amber Telfer implemented a first version of the VC linking 
and browsing. Phil Oh helped extend this into the current version of the certification 
assistant. 
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